Communicability-Inspired Positional Encoding (CIPE)

Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging. Such encodings should ideally induce an Attention-Compatible Geometry for self-attention: not merely describing graph structure, but defining a geometry whose inner pr…

View Post on Scour ↗

More from arXiv on Scour.

Rate this post to improve your feed:
🌰 Love  👍 Like  👎 Not for me